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1. RESEARCH DESIGN 



The central methodology of this study was a set of nine case studies of schools or other sites 
where technology was being used as a tool to support education reform. Hie case study 
approach is well*suited for building understandings about how a factor, such as the introduction 
of technology, affects a broader agenda of educational reform. We employed a multiple case 
method (see Greene & David, 1984), which; 

• Provides the researchers enough flexibility to adapt their questions and strategies for 
each site, while providing sufficiently comparable data for cross-site analysis. 

• Provides the researchers adequate exposure to sites for richly detailed descriptions of 
reform efforts involving technology at various levels that are useful for both 
practitioners and policy-makers. 

• Allows researchers to explore linkages between observed practice and potential 
explanatory factors, to d velop tentative hypotheses about causal factors, and to 
specify the conditions ui der which die observed relationships appear to hold true. 

The case studies focused on collecting information to answer the research questions displayed in 
Table 1. These questions were organized into the following topics: 

• General Context. Political/flscal/demographic setting for the technology application 
within the context of the state, district, school, and classroom. 

• Reform Context. Status of the site with respect to education reform initiatives (e.g., 
restructuring activities, curriculum reform, new ways of teaching and measuring 
student progress) and the role of the technology application in the site’s overall agenda 
for educational reform. 

• Design of the Technology Application. Development of the technology application, 
including incentives for the reform, the role of research in the design phase, 
involvement of teachers with software developers, the purpose of the technology 
application, and key characteristics of the approach. 

• Classroom Uses of Technology. Particular technology-supported activities or projects, 
at multiple grade levels and in multiple content areas, that illustrate the kind of 
student-centered instruction in our conceptual framework. 

• Implementation of the Technology Application. Factors that contributed or hindered 
the success of the application, role of policies and regulations at viuious levels of the 
system, resources needed (human and fiscal) to support the application, the integration 
of the technology application in teaching and learning, and the role of research in 
implementation. 

• Impact of the Technology Application. Evaluation strategies used to measure 
outcomes and the role of research in this process, demonstrated impact of the 
application on students and student learning, on teacher attitudes and behaviors, and 
other outcomes, including unanticipated benefits and costs. 
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2. METHODOLOGY 



Sample Selection 

The data collection portion of this project consisted of nine case studies of schools using 
technology as part of their education reform efforts, lire nine sites were chosen with an eye to 
representing the diversity of American schools and students and the very different paths to 
implementation that technology-supported innovations may take. At the same time, we sought 
sites whose use of technology was not for its own sake, but rather as a support for the kinds of 
instructional activities stressed in our conceptual model of educational reform. 

We collected ideas for potential case study sites through a review of the literature, 
discussions with practitioners and education technology experts at a national conference, from 
our project advisors, and from our own network of school and technology contacts. We 
conducted telephone interviews with over 30 potential sites to collect information regarding the 
criteria described below: 

• Potential for providing general lessons about the role of technology in educational 
reform. We looked for sites that could generate rich information about the design, 
implementation, and impact of technology applications in the context of educational 
reform. We gave priority to sites that api^ar^ to be engaged in a cohesive effort 
directed at improving education for all students. 

• Illustration of the roles of various players in education reform. To understand the 
roles of states, districts, and schools as well as those of the business community, 
parents, md foundations, v> tried to obtain a set of sites that represented variation in 
the set of major “players” involved in bringing about education reform. 

• Student population affected. Technology implementations aimed at promoting 
learning among economically disadvantaged students, and students of diverse ethnic 
and cultural backgrounds, were targeted. Technology is often used with disadvantaged 
students in ways that accentuate the differences between the instruction given the 
“haves” and the “have nots.” Although many of the more constructivist uses ol 
technology we were interested in have occulted most typically in schools serving 
relatively affluent populations, we sought out schools using technology in programs 
that challenge all students, including those whose backgrounds might have been 
regarded as putting them “at risk” of school failure in more traditional programs. 

• Stage of technology implementation. Design and implementation issues can best be 
addressed by studying sites in various stages of implementation. Although we wanted 
sites with enough experience to be able to draw some conclusions about what was and 
was not working, we arranged for variation in the schools’ length and intensity of 
interaction with technology. 

• Grade level focus. Since grade level affects the design, implementation, and impact of 
technology applications (reform has proved much more difficult at the secondary 
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school level), we made an effort to include middle and secondary settings as well as 
elementary schools within the site sample. 

Applying these criteria to the potential sites for which we had conducted phone interviews, 
we made recommendations to OERI and negotiated a final case study sample, described in 
Table 2. (The school names appearing in the table and throughout this report are pseudonyms.) 

Data Collection 

Preparation for Case Studies 

Preparation for the site visits entails several related procedures: notifying the sites that were 
approved for case study, obtaining and reviewing pertinent background information about these 
sites, developing sets of interview guides and debriefing forms, specifying codes to be used in 
preparing the data for qualitative analysis, and training data collectors. 

Notification of Sites. After obtaining OERI approval for the candidate sites, we informed 
the schools that they had been selected for study. Through our telephone interviews, we had 
already identified a point of contact for each site. The contact persons was asked to provide 
information on the process for obtaining permission to perform research in their school. We 
provided our contact with a written description of the purpose, of the study and the nature of the 
participation required of participating schools. We followed the schools’ individual procedures 
for obtaining permission to videotape students and teachers in selected classrooms (in some 
cases, schools had already obtained blanket permission for videotaping students in technology- 
using classes). Exhibit 1 contains a sample permission letter sent to parents. 

During the pre-visit phone discussions, we questioned our informant concerning the 
individuals outside the school (e.g., school district administrators, district technology 
cocifdinators, state agency representatives, union representatives, school board members, parents, 
consultants) who had a significant role in the reform effort and secured their help in setting up 
interviews with these individuals. 

Interview Guides. Interviews were one of the central methods of data collection for the site 
visits. We developed a set of interview guides to structure interviews with various types of 
respondents at each level for the site (district, school, and classroom) and to see that key data 
were collected from each (the guides appear at the end of this volume). These guides list the key 
topics that should be covered in each type of iruerview. We developed separate interview guides 
for school technology coordinate s, school principals, teachers, students, and district personnel. 
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Exhibit 1 



October 20, 1993 
Dear Parents, 

SRI International, a private non-profit research institute, has been funded by 
ihe Office of Educational Research to conduct a descdptive study of schools 
across the country that have used technology to support teaching and 
learning I its strong commitment to and work within 



During the week of December 6th, visitors from SRI will be observing 
classrooms and conducting informal interviews with teachers and students. 
The purpose of the observati ons is to learn how technology is used as a tool to 
supi^rt learning at WM The purpose of the interviews is to gain a better 
understanding of how a technology-supported curriculum "looks and works," 
through the eyes of teachers and students. Student interviews will be 
conducted in small groups, will list between twenty and thirty minutes, and 
participation will be voluntary. In addition to these observations and 
interviews, SRI visitors will be videotaping several classroom activities. The 
purpose of these videotapes is to demonstrate the kinds of classroom learning 
events that are made possible when teaching skill and technology are 
combined. The information gained through each of these data collection 
activities will be shared with others (e.g., researchers, educators, policy 
makers) who are interested in learning more about how technology can be 
used to support teaching and learning within schools. 

We are pleased to be able to include in a 

demonstration project which features exemplary educational programs. If you 
hav- : ^y questions or concerns about the project, or about your child's 

don in any of the datacoUectipn activities, please feel free to contact 



this area, 
project. 




has been selected for inclusion in this 




Sincerely, 



Kerry Olson 

Research Social Scientist 
SRI International 
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Although many of the same topics are covered in more than one guide, we anticipated correctly 
that the particular topics that would be appropriate to pursue and also the specific questions 
asked under each topic w luld vary from site to site, depending on the nature of the innovation 
and the organizational structure of the project. In cases weie judgments were collected (e.g., the 
effects of technology on student attitudes and learning), we covered the same topic witii multiple 
informants in order to get multiple perspectives. For more objective information (e.g., number of 
students in the school or project), we covered a given section of the interview guide with the 
most appropriate individuals and deleted them when interviewing others. 

Debriefing Forms and Preliminary Specification of Codes for Qualitative Analyses. The 
purpose of a debriefing form is to provide a standardized framework for writing a case study 
report. This is especially important when multiple sites and multiple researchers are involved. 

We used two debriefing forms for our study, one for schools and one for classrooms. The 
school-level debriefing form took a broad view that included a review of the educational context 
of the site; demographic information; educational indicators; history of educational reform at the 
site; levels of involvement at the district, state, and federal levels; history of technology 
applications, including incentives for use, when and how the applications started, technologies 
used, targ 't grades and curricula, key school players, and key outside players; overview of the 
way the technology is used by students and teachers; in.piementation details, including problems 
encountered, strategies for overcoming barriers, and facilitators and costs; impact of the 
technology use on students, teachers, and the school climate and processes; the way the 
technology use is evaluated; and respondents’ reflections and advice. 

The classroom debriefing form was similar in scope but focused on what was observed in the 
classroom during the site visit. Site visitors were prompted to write about the classroom context, 
features of reform that they observed, the classroom activities that took place, the technologies 
involved (e.g., microcomputers, wide area networks, hypermedia, animation, simulation), how 
the technologies wer^ used by students and teachers, and intended and actual benefits of the 
technology use from the perspective of students and teachers. The debriefing forms for our 
schools and classrooms are presented at the end of this volume. 

A special feature of our cross-case synthesis plan involved the use of software for qualitative 
data analysis, in our case, Ethnograph. The software facilitates the analysis process by searching 
for and retrieving data marked by code words or combinations of code words. It prints out text 
organized by the code or codes specified in a search procedure. The printout then can be 



assembled in a way that allows the researcher to read all the text pertaining to a particular topic, 
concept, or variable across all sites or a subset of sites. 

A critical step in using such software is the generation of a set of codes for labeling segments 
of text. We began the process of developing codes concurrently with designing the interview 
guides and debriefing forms. Figure 1 illustrates the way in which our predetermined set of 
codes ran through the interview guides, debrieHng forms, and subsequent analysis printouts. The 
debriefing forms parallel the interview guides in topic, prompting site visitors to synthesize and 
interpret their field notes along similar lines. This allows for smooth cross-site analysis. 

Site Visitor Training. The initial site visits, conducted during 1992-93 by the Project 
Director and Deputy Director, provided an opportunity to test out the realism of the proposed 
schedule, the ease of use of the data collection instruments, and the technical requirements for 
collecting video records on site. 

After refining interview guides and debriefing forms to facilitate the flow of interviews, these 
researchers set up and conducted training for the four additional researchers involved in on-site 
data collection. A day-long training session for site visitors was conducted in the fall of 1993. 
This training familiarized all site visitors with the purposes of the study and its conceptual 
framework, the interview guides and debriefing forms, the planned cross-case analysis and codes 
embedded in the debriefing forms, and the procedures to be used on site. Exhibit 2 contains the 
agenda for the training session. 

A video consultant was hired to instruct team members in the use of the Hi-8 video camera 
used on site and the special problems entailed in trying to capture shots of computer screens and 
to work under the highly variable lighting conditions of classrooms. 

Site Visits 

Review of Background Information. Before the site visits, site visitors reviewed the 
telephone interview record and all available materials pertaining to the design, implementation, 
and impact of the site’s technology and education reform activities. Documents included 
program descriptions, staff rosters, project proposals and budgets, brochures, videos, and 
evalu ition reports. This review provided site visitors with a preliminary framework for 
organizing the site visits and understanding the case. It also reduced burden on the sites by 
familiarizing site visitors with basic information. Valuable time on-site can then be spent 
verifying information, exploring key issues, and following up important leads. 
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Figure 1 Relationship between Interview Guides. Debriefing Form, and ETHNOGRAPH Printout 



Exhibit 2 



Technology and Education Reform 



Site Visitor Training 
November 1 , 1 993 
Agenda 



9:00-9:30 


Orientation to the Project 
Site Visit Schedule 
(distribute partial schedule) 


Barbara 


9:30- 10:15 


Site Visit Arrangements, Logistics & 
Sample Visit Schedule 

- Regular Visit 

- Repeat Visit 


Kerry 


10:15-10:30 


Equipment Checklist 


Kerry 


10:30 - 12:00 
and 

12:45 - 2:00 


Interview Guides 

- District Administrator 

- School Administrator 

- School Tech Coordtr 

- District Tech Coordtr 

- Student Groups 

- Teachers 

(show selected tapes) 


Barbara & Kerry 


2:00-2:30 


Video Logs & Dissemination Products 
(show AERA tape) 


Kerry 


2:30 - 3:30 


Videocam Training 


Consultant & Kerry 


3:30-4:30 


Case Study Reports & Analysis 

- School Debriefing Form 

- Classroom Debriefing Form 
(electronic templates to be 
distributed at a later time) 


Barbara 


4:30-5:00 


Introduction to Ethnograph & Draft of 
Codes for Analy 2 dng Data 
(distribute code sheets) 


Joan & Barbara 



t > 
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Site Visit Procedures. Each site visit was conducted by a two*person research team. The 
team approach has several advantages. From a methods perspective, it is desirable to have two 
researchers on-site so that one can take the lead role in interacting with respondents while the 
other plays a support role, in many cases operating the video equipment. This frees up the lead 
researcher to conduct interviews in a less structured and more personal manner, thus improving 
the quality of data collected. The presence of a second researcher also enables reliability checks 
and preliminary on-site analysis of the data as the researchers share their reactions and discuss 
tentative findings. Having two researchers available also lends flexibility to the scheduling 
process on-site; in some cases it was necessary for the researchers to split up in order to adhere to 
a pre-arranged interview schedule but still capture important classroom activities, the timing of 
which could not be anticipated in advance. 

The primary modes of data collection were observations and personal interviews. We 
interviewed administrators, teachers, students, parents, and other persons key to the design, 
implementation, or impact of the technology application, using the interview guides included at 
the end of this volume. 

We interviewed students in small groups to help them relax and bounce ideas off one another. 
After the first few visits, we developed found that useful insights could be gained also by 
interviewing individual students or student pairs as they demonstrated the technology on which 
they were working. These videotaped demonstrations gave us an opportunity to see student 
products and to probe for what they had learned from their interactions with the technology. 

At each site, we did initial brief observations of a broad range of classrooms in order to pick 
two classrooms for more intensive observation and videotaping.^ Our criteria for choosing these 
classrooms were a combination of the theoretical and the pragmatic — from the early discussions 
with administrators and teachers we tried to select classrooms that were using technology in tool- 
like ways to support complex, student-centered activities. At the same time, we were constrained 
by schedules, trying to select classrooms that would be doing something interesting on their 
technology- supported projects on the particular days we would be present to observe them. 

These more intensively studied classrooms were typically observed over repeated days, 
sometimes on multiple visits. More extensive interviews were conducted with the teacher or 
teachers, and typically, one of our two student focus groups was conducted with students drawn 
from this class. In addition, as our data collection proceeded, we found it useful to interview 

' The single exception was the “site” that was actually a network of over 50 schools. For this site, we visited four 
schools and selected a single electronic research class for more detailed observation and description. Hence, 
there were 17 clas.sroom activities in our final sample of detailed vignettes. 
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individual students or small groups as they developed or exhibited their technology-based work 
or demonstrated how they use particular pieces of software. We observed and videotaped 
classes, school activities, teacher meetings and training, and other key events related to 
technology use in these classrooms. 

In addition to the classroom-based data collection, we interviewed a wide range of other 
school respondents, including principals, project coordinators, and school technology 
coordinators. Moving out from the school, we then interviewed representatives of other 
institutions that were pivotal in the school’s opinion. These might include district personnel, 
researchers, representatives from business partners, leaders from parent groups, or education 
consultants. Our final selection of respondents depended on the school’s particular 
implementation history and its perception of the key players within it. For individual sites, we 
also interviewed a school board member, a union leader, and a state administrator. 

Throughout the site visit, the research team collected documents and other materials that 
supported our understanding of the technology application or the broader context of the site (e.g., 
videotapes, student achievement data). 

Schedule for Site Visits. We planned for an average of a week on site, but with the option to 
study more complex sites with more intensity, while still addressing key issues at all sites. Our 
methods were tested out in three initial site visits, conducted during the 1992-93 school year. 
Return visits to these three schools and the remainder of our site visits were conducted in the fall 
and spring of the 1993-94 school year. Exhibit 3 shows the schedule of activities for the initial 
visit to one site. We staggered the site visits over the period (avoiding December and the end of 
the school year, when a site visit would be more disruptive and school staff would be less 
available) to allow for an iterative process of data collection-synthesis-data collection-synthesis. 

Data Art.*^lysis 

Within-Site Analyses 

As mentioned above, preliminary analyses actually began on-site. The site visit team 
infoimally shared their perceptions with one another and began to relate and interpret 
information as they collected it. The two site visitors took responsibility as lead autlior on 
different sections of the debriefing foim. After lead authors produced first draft case reports for 
their assigned sections; the site visitors exchanged sections to review, critique, and augment each 
others work. This process was facilitated by access to the video and audiotape records of 
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Exhibit 3 



OERl T echnology and Reform in Education Project 
■■■A Site Visit 
May 18-21 



Overview of Activities 



Tuesday, May 18 

• Tour of school and overview of program 8:30 - 9:30 

• Teacher interviews 9:30 - 2:30 
J^gebra) 9:30 

(Arch & Design) 10:30 

J 8th Grade Research) 12:30 

— MUP Wb (Animation) 1:3 0 

Interview with former lead teacher) 3:00pm at 




Wednesday, May 19 

• Overview of Personal Growth Plan software 7:45 

• Classroom observations and videotaping 9:30m - 3:30 

Animation 9:30 
ILSLab 10:30 









Algebra 12:30 
8 th Grade Research 
Arch & Desist 2:30 
Interview with 
Interview with 
4:30 at 
Dinner with 
6:15 at 



1:30 





(Direct or) 3:00pm at Saturn 

(restr ucturi ng consultant) 

office (flHMHHipnS) 

(former Teacher Federation President) 



restaurant ini 



Thursday, May 20 

• Student interviews 9:30 - 2:30 

Animation students 9:30 
Arch & Design students 10:30 
Algebra students 12:30 
8th Grade R esearch students 1:30 

• Interview with Principal) 2:30 

• Parent interview 2:30 



Friday, May 21 

• Classroom observations and videotaping 9:30 -2:30 

Animation 9:30 
ILS Lab 10:30 
Algebra 12:30 
Video class 1: 30 

• Intervie w with W — ^ (formative evaluator) 

2:30 atWHH ride to airport 



A telephone interview has beenscheduledwith Assistant 

Superintendent of Teaching/flBHMBHHI^ at 8:30 am on June 3. 
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interviews and classroom activities. Selected interviews with key informants were transcribed to 
support the inclusion of direct quotation in the case study records. 

Use of Qualitative Analysis Software 

Developing Theme Codes. After the majority of write-ups were completed, the research 
team members read one another’s write ups and met as a group for a full day to share 
impressions and begin the process of interpreting the findings from a cross-site perspective. We 
began by focusing on individual cases ana then worked across cases. Our shared conceptual 
framework, exemplified by the debriefing form, helped to structure the discussion, but by this 
time, we were thinking beyond the debriefing framewoik to look for higher-order patterns and 
issues that we had not recognized when the debriefing forms were designed. We focused 
especially on successful sites and what made them so, and the apparent reasons some supposedly 
“exemplary” sites hadn’t turned out to be so exemplary after all. As we generated observations 
about our sites, we began to identify potential cross-site themes and corresponding theme codes. 
Exhibit 4 contains a f ull listing of theme codes as well as the heading codes from the debriefing 
forms. 

Data Coding. Once the debriefing forms were converted to ASCII files, the next step was the 
insertion of codes. Embedded heading codes were inserted on-line by clerical staff; theme coding 
and other more complicated coding was inserted by researchers on hard copy printouts and then 
inserted into the Ethnograph files by clerical staff. Exhibit 5 contains a section of a hand-coded 
text file. 

Once all the codes were inserted, searches were run. A search pulls out all segments of text 
th'it are coded with the code word being search 'd. (Exhibit 6 shows a sample search output.) As 
noted earlier, multiple code searches can be done on all files or a subset of files, so that researchers 
have limitless ways to explore the data. 

The search output then was organized in six major categories; 

• Technology Implementation 

• Technology Climate 

• Curriculum Content and Technology Usfs 

• Reform Features of Technology Use 
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Exhibit 4 



THEME CODES 



Code 

CORE 



GRASROOT 

POLITICS 

LEADER 

DICTATE 

EDGOALS 

MIS-MATCH 

FORETHOT 

HEADSTART 

PR 

PEER-LESS 

BAKESALE 

NO-FUNDS 

SUGAR 

NEW-ONLY 

CHANGE 

LINK 

MAGNET 

GREEN 

CONTENT 

RESEARCH 

WRITE 

SCIENCE 

MATH 

SOCSTUD 

LANGART 

SPEAK 



Use 

Pianning/Implementation Issues 

Assimilation of technology into the core instructional program, tool technologies, 
communication includes statements that tech is used primarily as a tool; integrated acr 
curriculum, etc. 

Teachers have been/are involved in tech planning and implementation; site based 
management 

Political climate of classroom, school, district that affect tech program in some way 
Presence of Leadership 

Top down approach; tech imposed from above (hierarchy) 

Plans for tech are built around education goals; includes ex. of tech not being selected 

(e.g., ILS) because it doesn’t fit with goals 

Mismatch between tech and education goals 

Thoughtful plan for equipment purchase/maintenance/upgrade 

Staff hired in advance to allow for careful planning (in new programs) 

Attention to community outreach, public awareness 

Needs of program are not due to the fact that school is ahead of others in tech 
use/reform 

Creative flmdraising/grant writing 

Lack of funds to support implementation 

Dependence on external factors 

Seed money mentality, no one pays for what is not new 

Lack of continuity in tech, leadership, facility; teacher turnover 

Increased linkages with other institutions; resources 

Increased role as magnet school 

Envy/jealousy on the part of other sites 

Curriculum Content 

Curriculum content is addressed 

Research skills 

Writing 

Science 

Mathematics 

Social Studies 

Reading/language arts other than composition 
Language (English, Spanish) 
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Exhibit 4 (Continued) 



H\t»ERMED 

WRITE 

NETWORK 

VIDEO 

SIMULATION 

TUTOR 

SEXPLORE 

SCOMM 

STOOL 

KIDUSE 

MASS 

MISER 

NORMS 

PARENT 



MULTIAGE 

REAL 

INTERD 

MINDSON 

DISTRACT 

KIDTEACH 

SCOLLAB 

SELFREG 

OUTINFO 

fflGHER 

CANDO 

THISTASK 

CREATE 

FRIENDLY 

MOTIV 

SSELF 

BilHAVE 

SHACKSKL 

PRODQUAI< 

QUALS 

FOLLOW 



Tech Uses 

Hypermedia 
Word processing 

Network applications; plans for network installation 

Video: videodisk 

Simulation 

Tutorial 

Exploratory software (e.g.. Simulations, CD research tools) 

T elecommunications 

Tool uses (word processing, database, graphics) 

Other 

Tech Climate 

Critical mass of teachers or administrators or others who are excited about/are using 
technology counter ex; few teachers excited about and using tech 
Reluctance to share tech or access 

Informal or formal rules that govern the way the technology is used 

Parent support of tech use; uses of tech by parents (e.g., take home computer program 

parent involvement 

Reform Features 

Multiage groupings 

Authentic projects; tech uses that mirror tech uses in the real world 
Interdisciplinary projects (also grasroot, assess, guide, duration, kidteach) 

Student Outcomes 

Students understand the instructional concepts that are involved in their technology us 
(similar to higher but does not necessarily involve critical thinking skills) 

Focus on surface features of tech (e.g., changing fonts rather that writing content 
Students teach the teacher; students teach students 
Student collaboration 
Self regulation 

Access to outside information sources 
Critical thinking skills 
More complex tasks 

Ability to accomplish certain tasks (usually related to content areas) 

Creativity 

Attention to audience 

Motivation 

Self esteem 

Time on task; fewer discipline issues 
Technology skills 
Quality of product 

Better prepared for life/jobs in 21st century/future jobs, schooling (often stated as reas 
for tech use) 

Lack of follow-up opportunities to use tech when student leaves school 
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Exhibit 4 (Continued) 



ASSESS 

REFEVAL 

NO-SHOW 



OPFORALL 



ACCESS 

LOCATION 

SCHED 

DURATION 

OWN 

WANTMORE 

THCKSKLS 

GUIDE 

NEWTEACH 

HARDTASK 

TCOLLAB 



TIMEUSE 

TSELF 

STAR 

SITETECH 

JIOHACK 

TIME 

NOTIME 

TRACE 

TRAINING 

TECHTA 

USETA 

HAVESKL 

BLOCK 

ONGOING 

MATURE 

PERSONAL 



Assessment 

Examples of or plans for authentic assessment (e.g., portfolio); tech use to support 
authentic assv^ss 

Any efforts to evaluate the reform and/or tech implementation (e.g., formative 
evaluation; surveys) 

Failure to show expected results (e.g., drop in test scores) 

Also in Debriefing; Stats, Account, Eval 

Access Issues 

Structure that gives all children a full range of technology experience; examples in whi 
tech is planned for use with disadvantaged kids, gender balance. [Counter ex: tech use 
with boys only, with few students.] 

Students are able to easily access technology for use on a regular basis (related to 
location). 

Distribution/location of tech to support access (e.g., lab/classroom/combinatioiv # of 
computers avail) 

Blocks of time for students to work on technology, flexible scheduling 
Duration of student projerts; extended versus short-term 

Teacher Outcomes 

Teachers internalize the technology, adapt for own use, make it theirs 
Want more tech, more training, eagerness to use tech 
Teacher’s tech skills 
Teacher as facilitator 

New instructional strategies; ways of teaching 
Assign more complex tasks 

Teacher collaboration, joint planning, teaching, training 

(Counter x; lack of collaboration, staff arrangements that pose barriers to working 
together) 

Use of time 
Teacher’s self esteem 

Opportunities for professional recognition (related to tself) 

Teacher Tech Training 

Technical resource person available on site 

i ack of on site tech support 

time to learn tech; time to plan tech use 

Lack of time for training; planning of tech use 

1 eachers are allowed to progress at own pace (can also be a negative) 

training provided (descriptions of training; may be specified as useta and techta) 

Training on tech 

Training focusing on use of tech in curriculum; support for tech integration with conte 

previous skills/knowledge/attitudes that support tech use 

Lack of familiarity/training/resistance to tech use 

Ongoing teacher training, follow-up, continued suppon of some type 

Lack of ongoing follow-up-up support 

Opportunity for teachers to take computer home for perscnal/professional use 
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Exhibit 4 (Conciuded) 



TCOMM 

TTOOL 

PRESENT 

MGT 

TCHUSE 

PICK 

GET 

WORK 

CRIME 



Teacher Uses of Tech 

Teiecommunications 

Tooi (word processing, database) 

Presentation (deiivery of instruction) 

Management (grading, monitoring student progress, attendance) 

Other 

Barriers to Tech Seiection/Purchase/Maintenance 

DifiRcuities seieaing hardware/software 

DifiBcuities obtaining software/hardware (includes lack of appropriate software) 

DifBculties maintaining hardware/software 

Vandalism (includes students destroying files on network) 



Coded Version of 



9/9/1994 15:43 Page 11 



and a video monitor to display in- 538 
school broadcasts, VCR, or videodisc 539 
presentations. 540 



Exhibit 5 



The availabilit^and range off 
equipment at has been 

experienced by the faculty as a 
challenge that, while broadening 
teaching and learning opportunities, 
is sometimes overwhelming and has not 
been utilized at its full potential. 
Teachers were not on board when 
decisions regarding specific hardware 
and software choices were made. (The 
original plan to hire the lead 
teaching staff well in advance, 
providing the needed time to plan the 
program and to become familiar with 
the technology, did not occur due to 
hiring limitations (the requirements 
for the positions restricted the 
number of applicants], as well as 
other factors [e.g., the lack of a 
facility, equipment delays].) 

In the words of one teacher, "this 
forced [the teaching staff] to have, 
use, and struggle with a technology 
that they might not have selecte d. ” 
Many of the teachers working at 
came to the school initially with 
little or no technological experience. 
Hired only six weeks prior to the 
opening of the school, the lead 
teaching staff was faced with a long 
list of priorities, such as developing 
the curriculum, organizing the 
schedule, and designing new approaches 
to monitoring and evaluating student 
progress. These critical issues, 
coupled with the daily 
responsibilities associated with 
teaching once the school was in 
session, took precedence over learning 
to use and integrate a variety of 
unfamiliar technologies. 

As in many other schools, th^mere 
presence of technology atfl^^BI 
no^ ensure its i mmediate adoption and 
in. igrated use. teachers 

.earned to use technology gradually, 
alongside their students. Additional 
technical training was provided 
through vendor-sponsored inservices. 




556 

557 

558 

559 

560 

561 

563 

564 

565 

566 
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SEARCH CODE OUTPUT 4/5/1995 16:23 

SORT CODE: TRAINING 



Page 23 



-i'5.2 Strategies 



SC: TRAINING 



# -TRAINING $-TIME 

: District Deroonstration School funds 

: provide for ten extra staff 

%-ASSESS 

development days. Sessions have 
focused on a range of issues, 
including technology use, 
collaborative learning, and authentic 
assess ment . Teachers pa rticipating in 
and have been 

provided with trainin^within the 



$-TIME 



context of these projects. The school 
has lengthened school days in order to 
end classes early every Wednesday for 
teacher planning and meeting time. 

The time is used for "circuit" 
meetings (to share and coordinate 
activities across grade levels) , 
parent conferencing, and special 
projects planning sessions 



fe-TCOL LAB 

■■■■■ ( teacher) : One of the things 
we have at that I really 

treasure as a new teacher is 
collaborative planning time that's set 
aside for us. We have minimum days 
every Wed... the first and the third 
Wednesday we split up into groups that 
areworlcingon different projects 

and we all get to 
collaborate with teachers who are 
doing things in their classroom that 
are similar to ours. And the second 
Wednesday we meet together as a 
circuit group, so the 456 grade 
teaci:ers all meet together. And the 
fourth ;’®dnesday we have time to work 
in our classrooms and to have parent 
conferences. This is the only school 
where I've worked, but I've talked to 
friends of mine who got their 
credentials with me who are completely 
isolated in their classrooms. As new 
teachers they don't have anyone to 
talk to about what should I do about 
this... or I want to teach a unit about 
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• Teacher Training and Outcomes 

• Student Outcomes. 

The search output filled four large notebooks. Within each notebook, the printout was further 
organized by code word. For example, the Technology Implementation notebooks encompassed 
29 codes beginning with HISTORY. Text coded with HISTORY was organized by site, after 
which the next set of printout would appear. This way of organizing the printout was selected 
because it enabled the researcher to read across sites while staying focused on a particular aspect of 
the technology application, in this case the history of each application. 

Cross-Site Analysis 

Cross-site analysis was begun informally at the interim debriefing when researchers shared 
their observations and initial inferences about each site. The final debriefing provided a context 
for more focused sharing, comparing and contrasting of findings, and discussion of cross-site 
themes. 

A senior researcher was assigned as principal author for each major topic to be addressed in 
the cross-site analysis. Her analysis was assisted by the output of the code searches by the 
Ethnograph qualitative analysis software, in the manner described above. Each researcher- 
author was able to conduct searches across the master database, focused on her specific issue. 

in addition to facilitating the consideraticn and elaboration of themes across the sites, 
Ethnograph also facilitated quick counts and status checks regarding the occurrence of selected 
variables within each of the sites. This provided an additional means for the researchers to verify 
and summarize their findings. For example, the researchers were quickly able to assess the 
number of classrooms within and across sites that reported specific intended benefits (e.g., 
h gher-order thinking skills) and teacher-observed effects (e.g., increased student motivation, 
greater collaboration) in relation to the integration of technology. Using software to assist in text 
analyses has several advantages (Ruskus, 1991): 

• The methodology provides a means of organizing voluminous qualitative data and 
conducting the analyse in a more systematic manner than would otherwise be 
possible. 

• The methodology accommodates the use of a highly detailed coding scheme that can 
be applied across sites and used by all site visit teams. This capability promotes 
consistent and uniform coding of the qualitative data. 




BEST COPY AVAILABLE 



• The methodology enables us easily to reconfigure the qualitative data for different 
audiences — local practitioners and state/federal policy-makers. 

• The record-keeping aspect of the methodology enables us to document qualitative 
themes with specific quotes or sets of quotes that are easily retrievable from the 
database. 
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7/31/93 

INTERVIEW GUIDE FOR SCHOOL ADMINISTRATOR 

A. Local Context and Overview of Reform Activities 

• School grades, size, demographic composition, % subsidized or free 
lunch, % LEP or NEP, % sp. ed., urbanicity, age of facility (if h.s., drop 
out rate & college entrance rate) 

• Education reform efforts underway and planned 

• Role of technology in school's reform efforts 

B. History of the Technology/Reform Effort 

• Most important technology applications in this school 

• How did application get started? (who, did what, when) 

• Incentives for the technology application 

C. Context for the Technology/Reform Effort 

• Involvement of district 

• Involvement of state 

• Involvement of business, consultants, parents, foundations 

• Other influences, including research 

D. Design Features of the Technology/Reform Application 

• Instructional goals & key features of approach 

• Target student population (s) 

« Target grade levels, curricula 

• Classes/teachers most affected 

E. Implementation Details 

• Integration into instruction 

« Sources of technical support 
« Impact of state and federal policies & regulations 

• Problems encountered in implementing the application 

• Strategies for overcoming barriers 

• Facilitators for implemeni ig the application 

• Inservice training/release time provided as part of the implementation 

• Cost estimate & resources involved (e.g., hardware, software, 

maintenance) 



Sch Admin 2 



F. Impacts of Technology/Reform Effort 

• Student/teacher roles 

• instructional practices 

• Curriculum 

• Classroom management 

• Student learning 

• Student motivation 

• Teachers' professional development 

• impact on administrator's role 

• Impact on school climate & schoolwide processes 

• Unanticipated benefits & costs 

• Any objective measures collected 

o External measures (i.e., district, state) & their influence on 
implementation 

G. Reflection & Dissemination 

• Other schools that have been helped to adopt this model 

• Advice for other schools 

• Future goals & di-ections 



n j 



o 



26 



8/14/93 

INTERVIEW GUIDE FOR SCHOOL TECH COORDINATOR 

A. Background and Role 

• Teaching & technology experience (self taught v. formal training) 

• Functions of the tech coordinator position 

B. Local Context and Overview of Reform Activities 

• Education reform efforts underway and planned 

• Role of technology in school’s reform efforts 

C. History of the Technology/Reform Effort 

• Most important technology applications in this school 

• How did application get started? (who, did what, when) 

• Incentives for the technology application 

D. Context for the Technology/Reform Effort 

• Involvement of district 

• Involvement of state 

• Involvement of business, consultants, parents, foundations 

• Other influences, including research 

E. Design Features of the Technoiogy/Reform Application 

• instructional goals & key features of approach 

• Target student population(s) 

• Target grade levels, curricula 

• Classes/teachers most affected 

F. Implementation Details 

• Integration into instruction 

• Technical support 

• Problems encountered in implementing the application 

• Strategies for overcoming barriers 

• Facilitators for implementing the application 

• Inservice training/release time provided as part of the implementation 

• Cost estimate & resources involved (e.g., hardware, software, 

maintenance) 
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SchTech 2 



G. Impacts of Technology/Reform Effort 

• Student/teacher roles 

• Instructional practices 

• Curriculum 

• Classroom management 

• Student learning 

• Student motivation 

• Teachers' professional development 

• Impact on administrator's role 

• Impact on school climate & schoolwide processes 

• Unanticipated benefits & costs 

• Any objective measures collected 

• External measures (I.e., district, state) & their influence on 

implementation 

H. Reflection & Dissemination 

• Other schools that have been helped tc adopt this model 

• Advice for other schools 

• Future goals & directions 
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INTERVIEW GUIDE FOR TEACHERS 



8/14/93 



A. Background 

• Specialization (grade, subject area) 

• Years at school & prior teaching experience 

• Previous experience with technology 

B. History of the Technology/Reform Effort 

• Description of development & Implementation at school level 

• Teacher's role in initiating, planning, & implementing 

• Perceptions regarding district, school, community support 

• Factors supporting implementation at school level 

• Factors hindering implementation at school level 

C. Implementation at Classroom Level 

• Overview of instructional environment before technology 

• Incentive for involving technology; instructional goal 

• Description of technology use 

• Student access; level of use 

• How technology use is managed & monitored 

D. Implementation Support 

• Participation in & perceived value of training related to technology use 

• Participation in & perceived value of training related to instructional uses 

of technology 

• Access to resources (e.g., equipment, software, time) 

• Opportunities for collegial support, feedback 

• Technical assistance and support 

• Recommendations for teacher development at other sites 

E. Impact of Reform & Technology Use on Instructional Practice 

• Instructional approach 

• Roles of teacher & students 

• Curriculum 

• Classroom management 

• Student assessment 
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Teachers 2 



F. Impact of Technology/Reform on Teacher Beliefs & Attitudes 

• How students learn 

• How & what teachers should teach 

• Value of communicating with other teachers 

• Self concept and status within teaching community 

• Unanticipated effects 

G. Impacts of Technology/Reform on Students 

• Student learning processes 

• Student skills & achievement 

• Motivation 

• Self concept 

• Social skills 

• Unanticipated effects 

H. Reflection & Dissemination 

• Current reform goals & tech, role in supporting them 

• Lessons learned regarding classroom implementation 
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INTERVIEW GUIDE FOR STUDENT GROUPS 



8/14/93 



A. Background 

• First names, grade levels, ages (ethnicity) 

• Previous experience with technology; tech exposure in home 

• S(>ecial technology interests 

B. Overview of Technology Use 

• Types of technology used in classroom 

• How each technology is used in classroom 

• Level of student access & participation 

• Observed differences in technology access/intersts (e.g., gender) 

• Technology training & support 

C. Technology’s Role & Value as a Tool to Support Learning 

Within specific subject area (writing, reading, science, math, art) 

• How is technology used? 

• How does it compare with paper-and-pencil or textbook-based 

activities? 

• Advantages and disadvantages 

• Individual preferences 




31 



0 u 



8 / 14/93 

INTERVIEW GUIDE FOR DISTRICT ADMINISTRATOR 

A. Local Context and Overview of Reform Activities 

• Political, fiscal, and demographic description of district 

(no. & level of schools in district; district enrollmont; no. or % eligible for 
free or reduced lunch; no. or % LEP nr NEP; no. or % special ed., ethnic 
composition of student body) 

• Status of school within the district 

• Education reform efforts underway and planned 

• District policies and plans for educational technology 

• Role of technology in reform efforts 

B. History of the Technoiogy/Reform Effort 

• How district got involved in the technology/reform effort 

• Overview of the development process 

• Incentives for the technology application 

C. Context for the Technology/Reform Effort 

• Involvement of state 

• Involvement of business, consultants, parents, foundations 

• Other influences, including research 

D. Design Features of the Technology/Reform Effort 

• Key features of the approach 

• Target student population(s) 

• Target curricula, grade levels, schools 

E. Implementation Details 

• Sources of technical support 

• Role of district actions, policies and regulations in implementation 

• Impact of state and federal actions, policies and regulations 

• Problems encountered in implementing the application 

• Strategies for overcoming barriers 

• Facilitators for implementing the application 

• Inservice training/release time provided as part of the implementation 

• Cost estimate & resources involved (e.g., hardware, software, 

maintenance) 



Distr Admin 2 



F. Impact of Technology/Reform Effort 

• District role in measuring impact 

• Measures used 

• Role of outside researchers or agencies in measuring impact 

• Impact on students (learning, motivation) 

• Impact on teachers (teaching styles, professional development) 

• Impact on school's status within the district 

• Impact on district climate & policies 

• Unanticipated benefits & costs 

• External measures (i.e., state) & their influence on implementation 

G. Reflections & Dissemination 

• Other schools/districts that you have helped to adopt this model 

• Advice for other districts 

• Future goals & directions 
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INTERVIEW GUIDE FOR DISTRICT TECH COORDINATOR 



8 / 22/93 



A. Local Context and Overview of Reform Activities 

• District-supported education reform efforts underv ay and planned 

• District policies and plans for educational technology 
’ Role of technology in reform efforts 

B. History of the Technology/Reform Effort 

« How district got involved in the technology/reform effort 

• Overview of the development process 

• Incentives for the technology application 

C. Context for the Technology/Reform Effort 

• Involvement of state 

• Involvement of business, consultants, parents, foundations 

• Other influences, including research 

D. Design Features of the Technology/Reform Effort 

• Instructional goals & key features of approach 

• T arget student population(s) 

• Target curricula, grade levels, schools 

E. Implementation Details 

• District role in technology planning & purchase 

• District role in in-service training 

• District role in providing technical support 

• Problems encountered in implementing the application 

• Strategies for overcoming barriers 

• Facilitators for implementing the application 

• Cost estimate & resources involved (e.g., hardware, software, 

maintenance) 





35 



District Tech 2 



F. Impact of Technology/Reform Effort 

• District role in measuring impact 

• Measures used 

• Role of outside researchers or agencies in measuring impact 

• Impact on students (learning, motivation) 

• Impact on teachers (teaching styles, professional development) 

• Unanticipated benefits & costs 

• External measures (i.e., state) & their influence on implementation 

G. Reflections & Dissemination 

• Other schools/districts that you have helped to adopt this model 

• Advice for other districts 

• Future goals & directions 
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10/28/93 

SCHOOL-LEVEL DEBRIEFING FORM 

[Note, ethnograph codes appear in bold italics for each section] 



Site: 

Address: 

Point of Contact: 

1.0 Summary [sum] 

Concise overview of the type of project and the main features of technology's 
contribution to education reform. Implementation issues illustrated by the 
case study should be highlighted. 

2.0 Context and Reform Overview 

2.1 Educational Context, [context ] Overview of the school including: 

• Setting (e.g., urban, rural, suburban) 

• Grade levels 

2.2 Demographic Information [demo] 

Circle applicable unit: School Class Project District Other: 
Number of certified teachers (FTE): 

Enrollment as of 9/93: 

Number or percent free or reduced lunch; 

Number or percent LEP or NEP: 

Number or percent special education: 

Ethnic Composition (Number or percent in each category) 

African American Hispanic Caucasian, Non-l-iispanc 

Native American AsiarVPacic Islander Other 



Estimated percent students w/ computers in home; 



2.3 Education indicators [sfafs] 

Summary of facts and statistics reflecting on how weli the school is 
functioning. Examples would include test scores, wait iists for entrance, 
dropout rat^^. 

2.4 Reform History [re1orm\. Summary of efforts toward restmctuiing 
or school reform, including a description of: 

• Nature and when started [history] 

• Reform Features [feature ] (circle all that apply) 

Nongraded groupings Extended periods Schoolwide themes 

Teacher professionalization Authentic assessment Use of outside resources 

• Key Individuals [key] 

• Funding sources for reform [funds] 

• District involvement/support [localref] 

• State/federal involvement/support [stateref] 

• Role of tech in reform efforts [techref] (classify and describe): 

Part of initial reform concept Emerging part of reform efforts Parallel effort 



3.0 History of Tech Applications 



3.1 Incentive for Technology Use [why] (what expected to accompiish) 

3.2 When & How Application Started [how] 

3.3 Technologies Used [types] (give numbers for hardware) 

• Microcomputers • Word processing 

• Spreadsheet 



ILS 

Other LANs 

WANs 

CDROM 



• Hypermedia 
Animation 
' Simulation 



• Interactive videodisc 

• Videocams and editing equipment 

• Other 
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3.4 Target Grades & Curricula [levels, curric] 

3.5 Key School Players [lnfleld\ 

(Provide names, titles, and a description of roles) 

3.6 Outside Players in Technology Application [outfield] 

Discuss important players outside the school including: 

• District involvement [distr] 

• State involvement [state] 

• Business involvement [bus] 

• Foundation involvement [foundtn] 

• Parent involvement [parent] 

• Other outside influences, including research 

4.0 Overview of Tech Uses 

4.1 Student Uses [kiduse]. Describe main ways in which students are 
using technology and classify uses as follows: 

• Tutorial 

• Exploratory 

• Communication 

• Tool 

4.2 Teacher Uses [tchuse]. Describe ways in which tech supports 
teacher functions and classify as follows: 

• Communication 

• Presentation 

• Instructional Management 
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5.0 Implementation Details 

5.1 Problems Encountered [probs]. Describe problem areas including 
any of the following that were issues according to teachers, tech 
coordinator, or administrators: 

• Difficulties in selecting hardware/software [pick] 

• Difficulties obtaining hardware/software [get] 

• Difficulties maintaining systems [work 

• Theft/vandalism [crime] 

• Teacher lack of knowledge/fear [block] 

• Lack of time for iearning to use technology or developing technology- 
reiated curriculum units [notIme] 

• Disappointing results from using technology [letdown] 

• Poor alignment of technology w/ instructional goals [mismatch] 

• Lack of funds [nofunds] 

• Resentment from nonparticipating classes/schools [green] 

• Lack of technical support [nohack] 

• Failure to see fast positive effects on test scores [noshow] 

> Other [bug] 

5.2 Strategies for Overcoming Barriers [solve]. Describe the steps 
taken to deal with the problems. Include any of the following: 

• Inservice training/release time 

• Involvement of outside experts 

• Creative fund raising 

• Other 



5.3 Facilitators [facll\. Describe factors that helped this project, including: 

• Technical support on hardware/software [techta] 

• Teaching skills (nontechnology) [havaski] 

• Technical assistance on integrating technology into instruction [instrta] 

• Supported time for teachers to develop skills [timesupp] 

• Other 

5.4 Costs [costs] (Estimates in dollars or person months) 

6.0 Impacts [lmpacf\ (Record objective and subjective reports from 
administrators, teachers, students, and parents.) 

6.1 Effects on Students [kldouf], include areas such as: 

• Motivation [motiv] 

• Selfesteem [kidestm] 

• Attendance, retention, disciplinary rates [kidbehav] 

• Technology skills [hackski] 

6.2 Effects on Teachers [tchout ]. Include areas such as: 

• inclination to assign complex tasks [hardtask] 

• New instructional strategies [tchstrat] 

• Self esteem and sense of professionalism [tchach] 

• Communication with other teachers [tchtalk] 

• Use of their time [timeuse] 

6.3 Effects on School Climate & Processes [schout] 

• increased linkages with outside organizations, resources [link] 

• Enhanced role as magnet [magnet] 
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6.4 External Accountability Measures & Their Influence [account] 

6.5 Evaluation [aval] {Describe ariy objective measures of effectiver)ess, ar)y 
formal evaluatior) reports, arid unanticipated ccsts & benefits.) 

• Test scores 

• Retention rates 

• Attendance 

• Discipline rates 

• Teacher turnover 

7.0 Reflection and Advice 

7.1 Dissemination [spread] (Has this model or the teachers’ expertise 
been transferred to any other sites?) 

7.2 Plans [plans] (Where does the school plan to go from here with its 
technology and reform efforts?) 

7.3 Advice [advice] (What advice would be offered to others 
contemplating similar efforts?) 
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CLASSROOM-LEVEL DEBRIEFING FORM 



Short Title: (Provide a short descriptive label for this use of technology) 
Teacher: 

Site: 



1.0 Summary [sum] 

Concise overview of this use of technology including what technology used, 
grade level of students, and subject matter involved. 

2.0 Classroom Context [context] 

Brief description of the classroom including anything striking about the physical 
layout, the number and grade level of students involved, the number teaching 
and support staff, teacher's background, and the way the instruction is 
organized (e.g., independent work, small groups, whole class). 

3.0 Reform Features [feature] (Classification of reform features 
illustrated.) 

Interdtedpinary Project-based Collaborative Learning Mixed Age 

Extended Activity Part of Core Program Teacher as Coach 

4.0 Description of Activity [activity] (Extended description of the 
purpose of the activity, subject areas involved, how it relates to curriculum 
goals, what teachers and students were doing, the length of both observed 
activities and other activities that are part of the same project and unit, and how 
technology was used.) 



5.0 Technologies involved [fyoes]: Check off technologies used. 



• Microcomputers 

• ILS 

• Other LANs 

• WANs 

• CDROM 

• interactive videodisc 

• Videocams and editing equipment 

• Other 



• Word processing 

• Spreadsheet 

• Hypermedia 

• Animation 

• Simulation 




6.0 Technology Use Classification and description of how technologies 
were used. 

• Student Uses [kid use] 

Tutorial [tutor] 

Exploratory [explore] 

Communication [commtn] 

Tool [tool] 

• Teacher Uses [tchuse] 

Communication 

Presentation 

instructional Management 

7.0 Intended Benefits of Technology Use [bene] include any of the 
following: 

• Peer-to-peer collaboration 

• Independent exploration 

• Access to outside information resources 

• Support for higher-level thinking 

• Creativity 

• Technology skills 

8.0 Realized Benefits of Technology [kidouf] Include any of the 

following: 

• Motivation [motiv] 

• Attendance, time on task, discipline [kidbehav] 

• Amount of collaboration & cooperative learning skills [coopig] 

• Ability to manage own learning [selfreg] 

• Access to outside information resources [outinfo] 

• Support for higher-level thinking [higher] 

• Complexity of task students can accomplish [taskacc] 

• Learning on specific tasks [kidlearn] 

• Creativity [create] 

• Greater attention to how a product will be understood by others [audience] 

• Technology skills [hackski] 
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